Vex : A Programming System for Extracting Content

This paper describes Vex: a programming system for extracting content information from unstructured media. Vex poses the problem of understanding media in terms of recognizing events in media streams. Vex approaches this problem by adapting computer science tools for matching patterns and parsing text. Vex allows the user to write an application directly in terms of patterns and actions. Vex transforms such a speciication into a program that examines a stream of video, recognizes when a speciied pattern of imagery has appeared, and performs the appropriate action. Vex addresses important issues in both multimedia computing and computer vision. As a multimedia tool, Vex is a language for penetrating opaque media. In computer vision, Vex represents an approach to building general purpose tools.